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AMENDMENTS TO THE CLAIMS: 

This listing of claims will replace all prior versions, and listings, of claims in the application: 
Listing of Claims: 

1. (Currently Amended) An interactive operation supporting system for an apparatus, 
comprising a display unit, a speech input unit, a speech output unit and an operation control unit, 
wherein said operation control unit inc lu d e s comprises : 

an assistant control means for generating a personified assistant and making said assistant 
appear on a screen of said display unit; 

an output speech control means for determining speech required for said assistant to 
output said assistant's speech to the outside through said speech output unit after speech 
synthesis; 

an input speech recognition means for recognizing user's voice as a speech inputted 
through said speech input unit; 

an interaction management means for managing interaction between said assistant and 
said user according to said assistant's speech determined by said output speech control means 
and said user speech recognized by said input speech recognition means; 

an ambient state generating means for enabling said personified assistant to act as if 
urging the user to input commands using the input speech recognition means when the 
interactive operating system is placed in a wait state; 

a connection means for connecting a television set/monitor to said system; 

a command interpreting means for specifying a user's intention or specifying said 
inputted user command based on a content of interaction traced by said interaction management 
means; and 
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a command execution means for executing function commands, said function commands 
including a channel selection or channel change command, 

wherein in response to said command interpreting means interpreting an inputted user 
command as a channel change command, said assistant control means places respective 
changeable broadcasting program display wi ndows in the shape of a ring around said assistant, 
and a part of the ring has a gap with no broadcasting program display window, and a 
broadcasting program display window of a program on a temporarily selected channel is placed 
at the forefront of the ring, 

wherein said output speech control means determines an assistant's speech based on a 
content of interaction managed by said interaction management means or an inputted user 
command specified by said command interpreting means, and 

wherein said output speech control means determines an assistant's speech for leading a 
user's intention when said command interpreting means fails to specify said user's intention or 
said inputted user command. 

2. (Original) The interactive operation supporting system according to claim 1, wherein 
said assistant control means determines an animation of said assistant based on a content of 
interaction managed by said interaction management means and/or an inputted user command 
specified by said command interpreting means. 

3-4. (Canceled) 
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5. (Original) The interactive operation supporting system according to claim 1, further 
comprising 

a connection means for connecting an external apparatus such as a television set/monitor 
and/or a video deck to said system, wherein said command interpreting means also interprets a 
command for controlling a function of said external apparatus including selection of a broadcast 
program channel and/or recording/reproducing of said broadcast program. 

6. (Original) The interactive operation supporting system according to claim 1 , further 
comprising: 

a communication means for connecting said system to a communication medium such as 
an external network or a telephone line, 

wherein said input speech recognition means recognizes audio data received via said 
communication medium. 

7. (Original) The interactive operation supporting system according to claim 1, further 
comprising: 

a communication means for connecting said system to a communication medium such as 
an external network and/or a telephone line; and 

a mail exchange means for making an exchange of electronic mails via said 
communication medium, 

wherein said output speech control means determines said assistant's speech based on a 
content of an incoming mail. 

Frommer Lawrence & Haug LLP 
745 Fifth Avenue 
New York, NY 10151 

212-588-0800 6 00493783 



U.S. Patent Application No. 09/9 1 1 , 1 09 

Response to Final Office Action of September 7, 2007 



PATENT 
450100-03355 



8. (Original) The interactive operation supporting system according to claim 1, wherein 
said interaction management means manages an input speech of a user as a message bound for 
another user, and said output speech control means determines said assistant's speech based on 
said message. 

9. (Original) The interactive operation supporting system according to claim 1, wherein 
said assistant control means places said personified assistant in a room scattered with various 
kinds of objects including a link to an information resource. 

10. (Original) The interactive operation supporting system according to claim 1, wherein 
said assistant control means places said personified assistant in a room scattered with various 
kinds of objects including a link to an information resource, and in response to an interest of said 
user in a recording medium including a link to a music content placed in said room, said 
command interpreting means interprets an inputted user command as a command for playing 
back said music content. 

1 1 . (Original) The interactive operation supporting system according to claim 1 , wherein 
in response to said command interpreting means interpreting an inputted user command, said 
assistant control means makes said assistant appear on a screen of said display unit. 

12. (Original) The interactive operation supporting system according to claim 1, further 
comprising 
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a connection means for connecting a television set/monitor to said system, and in 
response to said command interpreting means interpreting an inputted user command as a 
channel selection command, said assistant control means makes said assistant show a selected 
broadcasting program display window. 

13. (Canceled) 

14. (Currently Amended) The interactive operation supporting system according to claim 
claim 1 , wherein, in response to selection of a desired channel by moving said display 

window on said ring upon a channel change command from said user and said assistant control 
means also zooms up said selected broadcasting program display window. 

15. (Original) The interactive operation supporting system according to claim 1, further 
comprising: 

a connection means for connecting a secondary storage device for storing and/or 
reproducing a broadcasting program content to said system, 

wherein in response to said command interpreting means interpreting an inputted user 
command as a recorded program reproduction command, said assistant control means makes said 
assistant show a binder showing a view of recorded broadcasting program contents. 

16. (Original) The interactive operation supporting system according to claim 15, 
wherein in response to a selection for reproducing a recorded broadcasting program content, said 
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assistant control means zooms up said selected recorded broadcasting program content display 
window. 

17. (Original) The interactive operation supporting system according to claim 1, further 
comprising: 

a connection means for connecting a television set/monitor to said system, 
wherein in response to said command interpreting means interpreting an inputted user 
command as a channel change command, said assistant control means makes said assistant 
appear with a list of changeable broadcasting programs arranged in a matrix shape. 

18. (Original) The interactive operation supporting system according to claim 17, 
wherein in response to a selection of a desired channel, said assistant control means zooms up 
said selected broadcasting program display window. 

19. (Original) The interactive operation supporting system according to claim 17, 
wherein EPG distributed as part of data broadcast is applied for generating a list of broadcasting 
programs in a matrix form. 

20. (Original) The interactive operation supporting system according to claim 1, further 
comprising: 

a connection means for connecting a television set/monitor to said system; 
a communication means for connecting said system to a communication medium such as 
an external network and/or a telephone line; and 
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a mail exchange means for making exchange of electronic mails via said communication 
medium, 

wherein said assistant control means makes an incoming mail display image to appear on 
a screen of said display unit in response to acceptance of mail. 

21. (Original) The interactive operation supporting system according to claim 1, further 
comprising: 

a text conversion means for converting a Japanese ideogram Kanji related to text data 
displayed on a screen of said display unit into a corresponding phonetic character Kana. 

22. (Original) The interactive operation supporting system according to claim 1, further 
comprising: 

a communication means for connecting said system to a communication medium such as 
an external network or a telephone line; and 

a certifying means for certifying an information terminal connected to said system via 
said communication medium. 

23. (Original) The interactive operation supporting system according to claim 1, further 
comprising: 

a connection means for connecting a television set/monitor to said system; and 
an extraction means for extracting text information from a received broadcasting program 
content. 

Frommer Lawrence & Haug LLP 
745 Fifth Avenue 
New York, NY 10151 

212-588-0800 10 00493783 



U.S. Patent Application No. 09/91 1,109 

Response to Final Office Action of September 7, 2007 



PATENT 
450100-03355 



24. (Original) The interactive operation supporting system according to claim 23, 
wherein text information extracted by said extraction means is superimposed on a content of 
another broadcasting program on said screen. 

25. (Currently Amended) A method for supporting interactive operation of an apparatus 
or other externally connected apparatus including a display unit, a speech input unit and a speech 
output unit, comprising: 

an assistant control step for generating a personified assistant and making said generated 
assistant appear on a screen of said display unit; 

an output speech control step for determining a speech required for the assistant to output 
the assistant's speech to the outside through said speech output unit after speech synthesis; 

an input speech recognition step for recognizing a user's voice as a speech inputted 
through said speech input unit; 

an interaction management step for managing interaction between said assistant and said 
user according to said assistant's speech determined by said output speech control step and said 
user speech recognized by said input speech recognition step; 

an ambient state generating step for enabling said personified assistant to act as if urging 
the user to input commands using the input speech recognition step when the interactive 
operating system is placed in a wait state; 

a command interpreting step for specifying said user's intention or specifying said 
inputted user command based on a content of interaction traced by said interaction management 
step; 

a connecting step for connecting a television set/monitor to said apparatus; 
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a command executing step for executing function commands, said function commands 
including a channel selection or channel change command; 

a first determining step for determining an assistant's speech based on the content of 
interaction managed by said interaction management step or an inputted user command specified 
by said command interpreting step; [[and]] 

a second determining step for determining an assistant's speech for leading a user's 
intention when said command interpreting step fails to specify said user's intention or said 
inputted user command [[.]] ; and 

a positioning step for, in response to interpreting an inputted user command as a channel 
change command by the command interpreting step, placing respective changeable broadcasting 
program display windows in the shape of a ring around said assistant, and placing a gap with no 
broadcasting program display window as a part of the ring, and a placing broadcasting program 
dis play window of a program on a temporarily selected channel at the forefront of the ring. 

26. (Original) The method for supporting interactive operation according to claim 25, 
wherein said assistant control step determines an animation of said assistant based on a content 
of interaction managed by said interaction management step and/or an inputted user command 
specified by said command interpreting step. 

27-28. (Canceled) 

29. (Original) The method for supporting interactive operation according to claim 25, 
wherein said apparatus further includes a connection means for connecting an external apparatus 
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such as a television set/monitor and/or a video deck to said apparatus, and said command 
interpreting step also interprets commands for controlling a function of said external apparatus 
including selection of a broadcast program and/or recording/reproducing of said broadcast 
program. 

30. (Original) The method for supporting interactive operation according to claim 25, 
wherein said apparatus further includes a communication means for connecting said apparatus to 
a communication medium such as an external network and/or a telephone line, and said input 
speech recognition step recognizes audio data received via said communication medium. 

31. (Original) The method for supporting interactive operation according to claim 25, 
wherein said apparatus further includes a communication means for connecting said apparatus to 
a communication medium such as an external network and/or a telephone line, and a mail 
exchange means for making an exchange of electronic mails via said communication medium, 
said output speech control step determining said assistant's speech based on a content of an 
incoming mail. 

32. (Original) The method for supporting interactive operation according to claim 25, 
wherein said interaction management step manages an input speech of a user as a message bound 
for another user, and said output speech control step determines said assistant's speech based on 
said message. 
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33. (Original) The method for supporting interactive operation according to claim 25, 
wherein said assistant control step places said personified assistant in a room scattered with 
various kinds of objects including a link to an information resource. 

34. (Original) The method for supporting interactive operation according to claim 25, 
wherein said assistant control step places said personified assistant in a room scattered with 
various kinds of objects including a link to an information resource, and in response an interest 
of said user in a recording medium including a link to a music content placed in said room, said 
command interpreting step interprets an inputted command as a command for playing back said 
music content. 

35. (Original) The method for supporting interactive operation according to claim 25, 
wherein in response to said command interpreting step interpreting an inputted user command, 
said assistant control step makes said assistant appear on a screen of said display unit. 

36. (Original) The method for supporting interactive operation according to claim 25, 
wherein said apparatus further includes a connection means for connecting a television 
set/monitor to said apparatus, and in response to said command interpreting step interpreting an 
inputted user command as a channel select command, said assistant control step makes said 
assistant show a selected broadcasting program display window. 

37. (Canceled) 
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38. (Currently Amended ) The method for supporting interactive operation according to 
claim 37 claim 25 , wherein, in response to selection of a desired channel by moving said display 
window on said ring upon a channel change command from said user and said assistant control 
step also zooms up said selected broadcasting program display window. 

39. (Original) The method for supporting interactive operation according to claim 25, 
wherein said apparatus further includes a connection means for connecting a secondary storage 
device for storing and/or reproducing a broadcasting program content to said apparatus, and in 
response to said command interpreting step interpreting an inputted user command as a recorded 
program reproduction command, said assistant control step makes said assistant show a binder 
showing a view of recorded broadcasting program contents. 

40. (Original) The method for supporting interactive operation according to claim 39, 
wherein in response to a selection for reproducing a recorded broadcasting program content, said 
assistant control step zooms up said selected recorded broadcasting program content display 
window. 

41. (Original) The method for supporting interactive operation according to claim 25, 
wherein said apparatus further includes: 

a connection means for connecting a television set/monitor to said apparatus, and in 
response to said command interpreting step interpreting an inputted user command as a channel 
change command, said assistant control step makes said assistant appear with a list of changeable 
broadcasting programs arranged in a matrix shape. 
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42. (Original) The method for supporting interactive operation according to claim 41, 
wherein in response to a selection of a desired channel, said assistant control step zooms up said 
selected broadcasting program display window. 

43. (Original) The method for supporting interactive operation according to claim 41, 
wherein EPG distributed as part of data broadcast is applied for generating a list of broadcasting 
programs in a matrix form. 

44. (Original) The method for supporting interactive operation according to claim 25, 
wherein said apparatus further includes: 

a connection means for connecting a television set/monitor to said apparatus, a 
communication means for connecting said apparatus to a communication medium such as an 
external network and/or a telephone line, and a mail exchange means for making exchange of 
electronic mails via said communication medium, and said assistant control step makes an 
incoming mail display image to appear on a screen of said display unit in response to acceptance 
of mail. 

45. (Original) The method for supporting interactive operation according to claim 25, 
further comprising: 

a text conversion step for converting a Japanese ideogram Kanji related to text data 
displayed on a screen of said display unit into a corresponding phonetic character Kana. 
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46. (Original) The method for supporting interactive operation according to claim 25, 
further comprising: 

a communication step for connecting said apparatus to a communication medium such as 
an external network or a telephone line; and 

a certifying step for certifying an information terminal connected to said apparatus via 
said communication medium. 

47. (Original) The method for supporting interactive operation according to claim 25, 
wherein said apparatus further includes: 

a connection means for connecting a television set/monitor to said apparatus; and 
an extraction step for extracting text information from a received broadcasting program 
content. 

48. (Original) The method for supporting interactive operation according to claim 47, 
wherein text information extracted by said extraction step is superimposed on a content of 
another broadcasting program on said screen. 

49. (Currently Amended) A computer-readable storage medium storing a computer- 
executable program when executed, the computer-executable program causes a computer system 
for describing an interactive operation support pro c essing on a computer system, said to process 
an interactive operation support processing applied to of an apparatus includin g comprising a 
display unit, a speech input unit and a speech output unit for supporting input of a user command 
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to said apparatus and/or another externally connected apparatus, said program comprising the 
steps of: 

an assistant control step for generating a personified assistant to make said generated 
assistant appear on a screen of said display unit: 

an output speech control step for determining a speech required to said assistant for 
output of said assistant's speech to the outside through said speech output unit after speech 
synthesis; 

an input speech recognition step for recognizing a user's voice inputted through said 
speech input unit as speech; 

an interaction management step for managing interaction between said user and said 
assistant according to said assistant's speech determined by said output speech control step and 
said user speech recognized by said input speech recognition step; 

an ambient state generating step for enabling said personified assistant to act as if urging 
the user to input commands using the input speech recognition step when the interactive 
operating system is placed in a wait state; 

a connecting step of connecting a television set/monitor with said apparatus; 

a command interpreting step for specifying said user's intention or said inputted user 
command based on a content of interaction traced by said interaction management step; and 

a command executing step for executing function commands, said function commands 
including a channel selection or channel change command; 

a first determining step for determining an assistant's speech based on the content of 
interaction managed by said interaction management step or an inputted user command specified 
by said command interpreting step; 

Frommer Lawrence & Haug LLP 
745 Fifth Avenue 
New York, NY 10151 

212-588-0800 18 00493783 



U.S. Patent Application No. 09/91 1,109 

Response to Final Office Action of September 7, 2007 



PATENT 
450100-03355 



a second determining step for determining an assistant's speech for leading a user's 
intention when said command interpreting step fails to specify said user's intention or said 
inputted user command [[.]]; 

a positioning step for, in response to interpreting an inputted user command as a channel 
change command by the command interpreting step, placing respective changeable broadcasting 
program display windows in the shape of a ring around said assistant, and placing a gap with no 
broadcasting program display window as a part of the ring, and placing a broadcasting program 
display window of a program on a temporarily selected channel at the forefront of the ring. 

50. (Original) The interactive operation supporting system according to claim 1 , further 
comprising: 

a code conversion means for converting a character related to text data according to a 
group or system of characters or codes displayed on said display unit into a corresponding 
character of another group or system of characters or codes. 

51. (Original) The method for supporting interactive operation according to claim 25, 
further comprising: 

a code conversion step for converting a character related to text data according to a group 
or system of characters or codes displayed on said display unit into a corresponding character of 
another group or system of characters or codes. 

52. (Previously Presented) The interactive operation supporting system according to 
claim 1, wherein said motions or said act includes a leading question. 
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53. (Original) The interactive operation supporting system according to claim 1, wherein 
said ambient state generating means enables said personified assistant to enter said wait state 
with words and/or utterances encouraging a response from said user. 

54. (Original) The interactive operation supporting system according to claim 53, 
wherein said words and/or utterances encouraging a response include the words "Ah" and 
"Well." 

55. (Original) The interactive operation supporting system according to claim 1, wherein 
said ambient state generating means enables said personified assistant to enter said wait state 
with gestures encouraging a response from said user. 



Frommer Lawrence & Haug LLP 
745 Fifth Avenue 
New York, NY 10151 

212-588-0800 20 00493783 



